System and method for synthesizing magnetic resonance images

ABSTRACT

Methods and systems for synthesizing contrast images from a quantitative acquisition are disclosed. An exemplary method includes performing a quantification scan, using a trained deep neural network to synthesize a contrast image from the quantification scan, and outputting the contrast image synthesized by the trained deep neural network. In another exemplary method, an operator can identify a target contrast type for the synthesized contrast image. A trained discriminator and classifier module determines whether the synthesized contrast image is of realistic image quality and whether the synthesized contrast image matches the target contrast type.

CROSS REFERENCE

This application claims priority to U.S. Provisional Application No.62/631,102, filed Feb. 15, 2018, which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to magnetic resonance imaging(MRI), and more specifically, to synthetic MRI using a deep learningapproach.

BACKGROUND

MRI is a widely accepted and commercially available imaging modality forobtaining medical images of the interior of a patient based on magnetismof the nucleus. Various tissues, such as water-based tissues, fat-basedtissues, and fluids, in a human body can have different signalintensities on magnetic resonance (MR) images due to differences in MRproperties. The differences are described as image contrasts. MRI canproduce a wide range of contrasts by emphasizing a particular MRproperty while minimizing the others. For example, a protondensity-weighted image emphasizes the difference in spin density ofvarious tissues/fluids being analyzed. A T1-weighted image emphasizesthe difference in relaxation time for the recovery of magnetizationalong the longitudinal direction. A T2-weighted image emphasizes thedifference in relaxation time for the recovery of magnetization alongthe transverse direction. A short TI inversion recovery (STIR) imagesuppresses signals from fat. A fluid attenuated inversion recovery(FLAIR) image suppresses signals from fluid, and so on.

Different contrasts can be produced by using different pulse sequencesand choosing appropriate pulse sequence parameters. For example, aT1-weighted image can be produced by spin echo (SE) sequences orgradient echo (GRE) sequences with short echo time (TE) and shortrepetition time (TR). A T2-weighted image can be generated by using SEor GRE sequences with long TE and long TR. In many medical examinations,different image contrasts are needed for diagnosis, which are acquiredby performing several scans with different pulse sequences andparameters.

To reduce scanning time, a technology called synthetic MRI has beendeveloped which can reconstruct multiple image contrasts with a singlescan. With synthetic MRI, T1- and T2-weighted, T1- and T2-FLAIR, STIR,and/or proton density-weighted images can be produced from signalsacquired in one scan in far less the total time it would take to acquireeach contrast separately. Generally speaking, synthetic MRI uses amulti-delay multi-echo (MDME) sequence which includes interleavedslice-selective radio frequency (RF) pulse and multi-echo acquisition.This sequence is repeated with different delays between the RF pulse andthe acquisition. MR signals acquired with MDME sequences are used tocompute parameter maps (e.g., T1 map, T2 map, and proton density map,etc.) based on a pre-defined model which describes MR signal behaviorpixel by pixel (or voxel by voxel). The parameter maps andoperator-specified parameters (e.g., TE, TR, delay) are then used tocreate a “synthesized” image based on the model pixel by pixel (or voxelby voxel). The synthesized image is comparable to what would have beenformed by using the operator-specified parameters in an actual MR scan.

Clinical studies have shown the applicability of synthetic MRI for brainscans. However, the pixel-wise (or voxel-wise) model-fitting method asdescribed above may result in inaccurate parameter estimation andundesired artifacts because the model-fitting method tries to achieve aclosest fitting of parameters which may lead to artifacts and errors inthe synthesized images due to nonlinear nonlocal transform. For example,as shown in clinical studies, sequence-specific artifacts such asincomplete cerebrospinal fluid (CSF) suppression and pseudo-edgeenhancement are more pronounced in synthesized T2-FLAIR images. Inaddition, the model-fitting method cannot be used to synthesize certainimage contrasts such as T2*-weighted image, which reflectssusceptibility differences in tissues. Improvements to synthetic MRI aregenerally desired.

SUMMARY

In one embodiment, the present disclosure provides a method forsynthesizing an MR contrast image. The method comprises performing aquantification scan, using a trained deep neural network to synthesize acontrast image from a quantitative acquisition obtained by thequantification scan, and outputting the contrast image synthesized bythe trained deep neural network.

In another embodiment, the present disclosure provides an MRI system.The MRI system comprise a gradient coil assembly configured to generategradient magnetic fields, a radio frequency (RF) coil assemblyconfigured to generate RF pulses, a display, and a controller incommunication with the gradient coil assembly, the RF coil assembly, andthe display. The controller is configured to instruct the gradient coilassembly and RF assembly to generate a sequence perform a quantificationscan, instruct a trained deep neural network to synthesize a contrastimage from a quantitative acquisition obtained by the quantificationscan, and output the contrast image synthesized by the trained deepneural network at the display.

In yet another embodiment, the present disclosure provides a method formethod for synthesizing an MR contrast image. The method comprisesperforming a quantification scan, receiving a target contrast typeidentified by an operator, and using a trained deep neural network tosynthesize a contrast image from a quantitative acquisition obtained bythe quantification scan based on the target contrast type. The methodalso comprises using a trained discriminator to determine whether thesynthesized contrast image is of realistic image quality, and inresponse to determining that the synthesized contrast image is ofrealistic image quality, using a trained classifier to determine thecontrast type of the synthesized contrast image, determining whether thedetermined contrast type matches the target contrast type, and inresponse to determining that the determined contrast type matches thetarget contrast type, outputting the synthesized image.

BRIEF DESCRIPTION OF THE DRAWINGS

Various aspects of this disclosure may be better understood upon readingthe following detailed description and upon reference to the drawings inwhich:

FIG. 1 is a schematic diagram of an MRI system, in accordance with anexemplary embodiment;

FIG. 2 is a schematic diagram of a system for synthesizing contrastimage(s) from a quantitative acquisition, in accordance with anexemplary embodiment;

FIG. 3 is a schematic diagram of a single block of a pulse sequence forperforming a quantification scan, in accordance with an exemplaryembodiment;

FIG. 4 shows T2-FLAIR images of a brain obtained by various techniques;

FIG. 5 is a schematic diagram of a system for synthesizing contrastimage(s) from a quantitative acquisition, in accordance with anotherexemplary embodiment;

FIG. 6 is a flow chart of a method for training the deep neural networkused in the system of FIG. 5 , in accordance with an exemplaryembodiment;

FIG. 7 shows T1-weighted and T2-FLAIR images of a brain obtained byvarious techniques; and

FIG. 8 is a flow chart of a method for synthesizing contrast image(s)from a quantification acquisition, in accordance with an exemplaryembodiment.

The drawings illustrate specific aspects of the described components,systems and methods for synthesizing MR images using a deep learningapproach. Together with the following description, the drawingsdemonstrate and explain the principles of the structures, methods, andprinciples described herein. In the drawings, the thickness and size ofcomponents may be exaggerated or otherwise modified for clarity.Well-known structures, materials, or operations are not shown ordescribed in detail to avoid obscuring aspects of the describedcomponents, systems and methods.

DETAILED DESCRIPTION

One or more specific embodiments of the present disclosure are describedbelow in order to provide a thorough understanding. These describedembodiments are only examples of the systems and methods forsynthesizing contrast MR images from a quantification scan using a deeplearning approach. The skilled artisan will understand that specificdetails described in the embodiments can be modified when being placedinto practice without deviating the spirit of the present disclosure.

When introducing elements of various embodiments of the presentdisclosure, the articles “a,” “an,” and “the” are intended to mean thatthere are one or more of the elements. The terms “first,” “second,” andthe like, do not denote any order, quantity, or importance, but ratherare used to distinguish one element from another. The terms“comprising,” “including,” and “having” are intended to be inclusive andmean that there may be additional elements other than the listedelements. As the terms “connected to,” “coupled to,” etc. are usedherein, one object (e.g., a material, element, structure, member, etc.)can be connected to or coupled to another object regardless of whetherthe one object is directly connected or coupled to the other object orwhether there are one or more intervening objects between the one objectand the other object. In addition, it should be understood thatreferences to “one embodiment” or “an embodiment” of the presentdisclosure are not intended to be interpreted as excluding the existenceof additional embodiments that also incorporate the recited features.

Referring to the figures generally, the present disclosure is related tosynthetic MRI. Synthetic MRI can reconstruct multiple image contrasts(e.g., T1- and T2-weighted, T1- and T2-FLAIR, proton density-weighted,STIR images) from MR signals acquired with a quantification sequence(e.g., MDME sequences) in a single scan. Conventionally, a pre-definedmodel describing MR signal behavior is used to compute parameter maps(e.g., T1, T2, and proton density maps) from the acquired MR signalspixel by pixel (or voxel by voxel). Then various image contrasts areproduced by synthesizing the parameter maps and operator-specifiedparameters (e.g., TE, TR, delay) based on the model pixel by pixel (orvoxel by voxel). However, the pixel-wise (or voxel-wise) model-fittingmethod does not produce satisfying results for some image contrast(e.g., T2-FLAIR), and furthermore, cannot be used to synthesize certainimage contrast (e.g., T2*-weighted image).

The present disclosure uses a deep learning based approach forsynthesizing MRI contrasts. In particular, a deep learning model (e.g.,deep neural network) is trained by a plurality of datasets, each datasetincluding one or more ground truth images each acquired by a contrastscan and a corresponding acquisition by a quantification scan. As usedherein, a “contrast scan” refers to the measurement of MR signalsreflecting difference (i.e., contrast) in physical parameters of tissuesbeing scanned. In an image acquired by a contrast scan, the absolutesignal intensity has no direct meaning; it is rather the intensitydifference that leads to a diagnosis. Contrast images are produceddirectly from contrast scans, for example, T1- and T2-weighted images,T1- and T2-FLAIR images, proton density-weighted images, STIR images,etc. are contrast images. As used herein, a “quantification scan” refersto the measurement of MR signals reflecting absolute values of physicalparameters of tissues being scanned. The deep learning model is trainedto map the acquisition by a quantification scan to the corresponding oneor more contrast images. “Acquisition by a quantification scan” is alsoreferred to as “quantitative acquisition” in this disclosure. Thetrained (and validated) deep learning model is then applied to a newquantitative acquisition to synthesize one or more contrast images. Thedeep learning based approach is shown to outperform the conventionalmodel-fitting method with improved synthesis accuracy and reducedartifacts. Furthermore, the deep learning based approach is capable ofsynthesizing certain image contrast (e.g., T2*-weighted image) whichcannot be produced by the conventional method.

In some embodiments, the deep learning model is trained in anintra-contrast manner. For example, one model is dedicated to aparticular contrast type or a separate channel is set up for eachcontrast type within a model, each model or channel being trainedseparately. In some embodiments, the deep learning model is trained inan inter-contrast manner. For example, one set of network parameters(e.g., weights and biases) is trained and used for multiple contrasttypes. Correlation between different contrast types is utilized to reacha mutual correction effect.

Referring to FIG. 1 , a schematic diagram of an exemplary MRI system 100is shown in accordance with an embodiment. The operation of MRI system100 is controlled from an operator workstation 110 that includes aninput device 114, a control panel 116, and a display 118. The operatorworkstation 110 is coupled to and communicates with a computer system120 that enables an operator to control the production and viewing ofimages on display 118. The computer system 120 includes a plurality ofcomponents that communicate with each other via electrical and/or dataconnections 122. The components of the computer system 120 include acentral processing unit (CPU) 124, a memory 126, which may include aframe buffer for storing image data, and an image processor 128. In analternative embodiment, the image processor 128 may be replaced by imageprocessing functionality implemented in the CPU 124. The computer system120 may be connected to archival media devices, permanent or back-upmemory storage, or a network. The computer system 120 is coupled to andcommunicates with a separate MRI system controller 130.

The MRI system controller 130 includes a set of components incommunication with each other via electrical and/or data connections132. The components of the MRI system controller 130 include a CPU 131,a pulse generator 133, which is coupled to and communicates with theoperator workstation 110, a transceiver 135, a memory 137, and an arrayprocessor 139. In an alternative embodiment, the pulse generator 133 maybe integrated into a resonance assembly 140 of the MRI system 100. TheMRI system controller 130 is coupled to and receives commands from theoperator workstation 110 to indicate the MRI scan sequence to beperformed during a MRI scan. The MRI system controller 130 is alsocoupled to and communicates with a gradient driver system 150, which iscoupled to a gradient coil assembly 142 to produce magnetic fieldgradients during a MRI scan.

The pulse generator 133 may also receive data from a physiologicalacquisition controller 155 that receives signals from a plurality ofdifferent sensors connected to an object or patient 170 undergoing a MRIscan, such as electrocardiography (ECG) signals from electrodes attachedto the patient. And finally, the pulse generator 133 is coupled to andcommunicates with a scan room interface system 145, which receivessignals from various sensors associated with the condition of theresonance assembly 140. The scan room interface system 145 is alsocoupled to and communicates with a patient positioning system 147, whichsends and receives signals to control movement of a patient table to adesired position for a MRI scan.

The MRI system controller 130 provides gradient waveforms to thegradient driver system 150, which includes, among others, Gx, Gy and Gzamplifiers. Each Gx, Gy and Gz gradient amplifier excites acorresponding gradient coil in the gradient coil assembly 142 to producemagnetic field gradients used for spatially encoding MR signals during aMRI scan. The gradient coil assembly 142 is included within theresonance assembly 140, which also includes a superconducting magnethaving superconducting coils 144, which in operation, provides ahomogenous longitudinal magnetic field B₀ throughout an open cylindricalimaging volume 146 that is enclosed by the resonance assembly 140. Theresonance assembly 140 also includes a RF body coil 148 which inoperation, provides a transverse magnetic field B₁ that is generallyperpendicular to B₀ throughout the open cylindrical imaging volume 146.The resonance assembly 140 may also include RF surface coils 149 usedfor imaging different anatomies of a patient undergoing a MRI scan. TheRF body coil 148 and RF surface coils 149 may be configured to operatein a transmit and receive mode, transmit mode, or receive mode.

An object or patient 170 undergoing a MRI scan may be positioned withinthe open cylindrical imaging volume 146 of the resonance assembly 140.The transceiver 135 in the MRI system controller 130 produces RFexcitation pulses that are amplified by an RF amplifier 162 and providedto the RF body coil 148 and RF surface coils 149 through atransmit/receive switch (T/R switch) 164.

As mentioned above, RF body coil 148 and RF surface coils 149 may beused to transmit RF excitation pulses and/or to receive resulting MRsignals from a patient undergoing a MRI scan. The resulting MR signalsemitted by excited nuclei in the patient undergoing a MRI scan may besensed and received by the RF body coil 148 or RF surface coils 149 andsent back through the T/R switch 164 to a pre-amplifier 166. Theamplified MR signals are demodulated, filtered and digitized in thereceiver section of the transceiver 135. The T/R switch 164 iscontrolled by a signal from the pulse generator 133 to electricallyconnect the RF amplifier 162 to the RF body coil 148 during the transmitmode and connect the pre-amplifier 166 to the RF body coil 148 duringthe receive mode. The T/R switch 164 may also enable RF surface coils149 to be used in either the transmit mode or receive mode. Theresulting MR signals sensed and received by the RF body coil 148 aredigitized by the transceiver 135 and transferred to the memory 137 inthe MRI system controller 130.

An MR scan is complete when an array of raw k-space data, correspondingto the received MR signals, has been acquired and stored temporarily inthe memory 137 until the data is subsequently transformed to createimages. This raw k-space data is rearranged into separate k-space dataarrays for each image to be reconstructed, and each of these separatek-space data arrays is input to the array processor 139, which operatesto Fourier transform the data into arrays of image data.

The array processor 139 uses a known transformation method, mostcommonly a Fourier transform, to create images from the received MRsignals. These images are communicated to the computer system 120 wherethey are stored in memory 126. In response to commands received from theoperator workstation 110, the image data may be archived in long-termstorage or it may be further processed by the image processor 128 andconveyed to the operator workstation 110 for presentation on the display118.

In various embodiments, the components of computer system 120 and MRIsystem controller 130 may be implemented on the same computer system ora plurality of computer systems.

Referring to FIG. 2 , a schematic diagram of a system 200 forsynthesizing contrast image(s) from a quantitative acquisition is shown,in accordance with an exemplary embodiment. At the training stage of adeep neural network 220, a quantitative acquisition (e.g., imagesacquired from a quantification scan) 210 is input to the deep neuralnetwork 220, which in turn outputs a synthesized contrast image 240. Thesynthesized contrast image 240 is compared with a corresponding groundtruth contrast image 250 acquired from a contrast scan. The difference(i.e., the loss) 245 is back projected to the deep neural network 220 tooptimize (i.e., to train) the network parameters. After being trained(and validated), the deep neural network 220 can be exploited to map aquantitative acquisition to corresponding contrast image(s), using theoptimized network parameters. The trained deep neural network 220 may bestored at the imaging device (e.g., memory 126 in FIG. 1 ), at an edgedevice connected to the imaging device, a cloud in communication withthe imaging device, or any appropriate combination thereof.

Referring to FIG. 3 , a schematic diagram of a single block of a pulsesequence 300 for performing a quantification scan is shown, according toan exemplary embodiment. A two-dimensional (2D) fast spin echo (FSE)multi-delay multi-echo (MDME) sequence may be performed for thequantification scan, which includes an interleaved slice selectivesaturation RF pulse and multi-echo acquisition. The saturation acts on aslice n, whereas the acquisition acts on a slice m, using the 90°excitation pulse and multiple 180° pulses. n and m are different slices.As such, the effective delay time between saturation and acquisition ofeach particular slice can be varied by the choice of n and m. In someembodiments, four (4) different choices of n and m are performed,resulting in four different delay times. The number of echoes may be setas two (2), at two different echo times. Hence, the result of thequantitative acquisition 210 is eight (8) complex images per slice(although two images are shown in FIG. 1 ). It should be understood thatthe example of 4 delays at 2 echoes is described herein forillustration, not for limitation. Any appropriate combination of delaysand echoes can be used to perform the quantification scan. It shouldalso be understood that any appropriate quantification sequence besidesMDME sequence may be used. Furthermore, the quantitative acquisition 210may be raw MRI signals in k-space in some embodiments, such as raw MDMEMRI.

Referring back to FIG. 2 , the deep neural network 220 may be amulti-scale U-Net convolutional neural network (CNN) with anencoder-decoder structure. In the example shown in FIG. 2 , an inputlayer 221 receives the quantitative acquisition 210. In someembodiments, the input layer 221 includes multiple channels toaccommodate the input of the multiple images as the result of aquantitative acquisition at once. In some embodiments, each of themultiple images is split up into real and imaginary components andstacked as a corresponding channel. A pooling layer 222 is configured todown-sample the output from the input layer 221.

The output from the pooling layer 222 is passed through three (3)encoder modules (223, 225, and 227) followed by three (3) decodermodules (232, 234, and 236), each module consisting of three (3)convolutional layers. In some embodiments, a down-sampling layer or anup-sampling layer is located at the end of a module for connection tothe following module. Each layer is made up of a plurality of “neurons”(also known as “nodes”) with trainable weights and biases. Each neuronreceives several inputs, takes a weighted sum over them, passes itthrough an activation function, and responds with an output. In someembodiments, concatenated connections and residual connections are usedto accelerate training convergence, improve the reconstruction accuracy,and restore resolution information.

The weights and biases of the convolutional layers in the deep neuralnetwork 220 are learned during training. More specifically, a lossfunction 245 is defined to reflect the difference between the contrastimage 240 synthesized by the deep neural network 220 and a correspondingground truth contrast image 250. The ground truth contrast image 250 maybe, for example, T1- or T2-weighted, T1- or T2-FLAIR, protondensity-weighted, STIR image, or the like, acquired from a contrastscan. In some embodiments, the loss function 245 is the mean-absoluteerror, i.e., equal to the mean of absolute difference of pixel valuesbetween the ground truth contrast image 250 and the synthesized contrastimage 240. In some embodiments, the synthesized contrast image 240 isregistered to the ground truth contrast image 250 before the calculationof the loss function 245. Other loss functions may be used, such asroot-mean-squared-error (RMSE), structural-similarity-index (SSIM), etc.The loss 245 is then back projected to the deep neural network 220 toupdate the weights and biases of the convolutional layers. A pluralitypairs of quantitative acquisition and corresponding contrast image(s)may be used to train the deep neural network 220. In this example, thedeep neural network 220 is trained in an intra-contrast manner. Forexample, one model is dedicated to a particular contrast type or aseparate channel is set up for each contrast type within a model, eachmodel or channel being trained separately.

The deep neural network 220 can be trained to synthesize a T2*-weightedimage by using ground truth T2*-weighted images as reference during thetraining. Generally, T2*-weighted images cannot be synthesized withconventional model-fitting method.

It should be understood that the layout of the neural network 220 asshown in FIG. 2 is for illustration, not for limitation. A differentneural network architecture may be used herein for synthesizing contrastimage(s) from a quantitative acquisition. For example, different neuralnetwork structures may include residual networks (ResNets), autoencoder,recurrent neural networks, and fully connected networks.

Referring to FIG. 4 , T2-FLAIR images of a brain obtained by varioustechniques are shown for comparison. The first row shows a ground truthT2-FLAIR image acquired by a contrast scan as the reference image and azoom-in view of the central region marked in the reference image. Thesecond row shows a T2-FLAIR image synthesized from a MDME acquisitionwith conventional model-fitting method, a zoom-in view of the centralregion, and a 5× magnified error map reflecting the error in the centralregion. The third row shows a T2-FLAIR image synthesized from a MDMEacquisition with a trained deep neural network as described above, azoom-in view of the central region, and a 5× magnified error mapreflecting the error in the central region. The MDME acquisition was a2D FSE based MDME with four (4) saturation delays and two (2) echoes.The deep neural network was trained on around 1900 different datasets,which include paired inputs and outputs at 26-30 planes from 8 subjectswith 8 types of data augmentation using rigid transformation to avoidoverfitting. Conventional 2D FSE brain images were used as ground truthcontrast images. Then the trained neural network was applied to otherdatasets, and the performance of the trained deep learning model wasevaluated based on the result.

T2-FLAIR, as discussed above, is one of the most challenging contraststo accurately synthesize in clinical studies. Conventional model-fittingmethod often results in recognizable artifacts which prevent itsclinical application. As shown in FIG. 4 , the data-driven deep learningapproach achieved both improved accuracy and reduced artifacts forsynthesizing T2-FLAIR. In particular, the zoom-in views demonstrate thatthe deep learning approach resolved the fake boundary artifacts whichappeared in the model-fitting result. The arrows highlight the reducedboundary artifacts. The error maps demonstrate that the deep learningapproach achieved lower errors than the conventional model-fittingmethod.

In addition, various metrics were used to compare the performance of thedeep learning approach with the performance of the conventionalmodel-fitting method, including for example, root-mean-squared-error(RMSE), peak-signal-to-noise-ratio (PSNR), and SSIM. All demonstratedsuperior performance of the deep learning approach over themodel-fitting method. With T2-FLAIR synthesis as an example, the deeplearning approach outperformed existing model-fitting based method.

Referring to FIG. 5 , a schematic diagram of a system 500 forsynthesizing one or more contrast images from a quantitative acquisitionis shown, in accordance with another exemplary embodiment. Aquantitative acquisition (e.g., images acquired by a quantificationscan) 510 and a target contrast type 515 are input to a deep neuralnetwork 520, which in turn outputs a synthesized contrast image 540. Thesynthesized contrast image 540 is then passed through a discriminatorand classifier module 550, which outputs a discriminator 552 and aclassifier 554. The discriminator 552 indicates whether the synthesizedcontrast image 510 is of realistic image quality. If so, the classifier554 indicates which contrast type the synthesized contrast image 510 is.Different than the system 200 of FIG. 2 , in this example, the deepneural network 520 is trained in an inter-contrast manner. For example,one set of network parameters is trained on and applied to multiplecontrast types. Thus, correlation between different contrast types isutilized to reach a mutual correction effect.

Like the quantitative acquisition 210 in FIG. 2 as discussed above, thequantitative acquisition 510 may be multiple complex imagesreconstructed from MR signals acquired by an MDME sequence. The targetcontrast type 515 may be identified by an operator (e.g., aradiologist). In some embodiments, a mask vector is used to indicate thetarget contrast type, using “1” for the desired contrast type in thevector while using “0” for the others.

Like the deep neural network 220 in FIG. 2 as discussed above, the deepneural network 520 may be a U-Net CNN with an encoder-decoder structure.In some embodiments, concatenated connections and residual connectionsare used to accelerate training convergence, improve the reconstructionaccuracy, and restore resolution information. The weights and biases ofthe convolutional layers are learned during training.

The discriminator and classifier module 550 may be another CNN includingmultiple convolutional layers. The module 550 is configured to determinewhether an image has realistic quality and if so, which contrast typethe image is. In some embodiments, two separate modules may be used asthe discriminator and the classifier, respectively. In operation, thediscriminator and classifier module 550 may be trained by a plurality ofdatasets. The training datasets may include realistic images of alltypes (T1- and T2-weighted, T1- and T2-FLAIR, proton density weighted,STIR, etc.) and synthesized images not of realistic quality. Each imagemay be labeled as real or fake. For images of realistic quality, thecontrast type can be further labeled. After the discriminator andclassifier module 550 is trained, it may be connected to the deep neuralnetwork 520 to help train the network 520.

Referring to FIG. 6 , a flow chart 600 of a method for training the deepneural network 520 is shown, in accordance with an exemplary embodiment.The datasets for training the network 520 may include a plurality pairsof quantitative acquisition and corresponding ground truth contrastimages acquired from contrast scans as references. In some embodiments,the training process can handle situations where different datasets havedifferent subsets of references. For each ground truth contrast image,the contrast type is labeled. Target contrast types(s) may be input tothe deep neural network 520 with the quantitative acquisition 510. Basedon the quantitative acquisition 510 and the target contrast type 515,the deep neural network 520 produces the synthesized contrast image 540,which is passed to the discriminator and classifier module 550.

At 602, the discriminator and classifier module 550 decides whether theimage 540 has realistic image quality. If the discriminator 552indicates “NO,” this indication is then back projected to the deepneural network 520 to update the network parameters, at operation 610.If the discriminator indicates “YES,” the method proceeds to 604, wherethe discriminator and classifier module 550 decides which contrast type554 the synthesized image 540 is. At 606, it is determined whether thecontrast type 554 matches the target contrast type 515. If it is “NO” atoperation 606, this indication is then back projected to the deep neuralnetwork 520 to update the network parameters, at operation 610. If it is“YES” at operation 606, the method proceeds to 608, where a loss iscalculated. The loss may be any appropriate function reflecting thedifference between the synthesized image 540 and the correspondingground truth contrast image of the same type. The loss is then backprojected to the deep neural network 520 to adjust the networkparameters, at 610. After being trained (and validated), the deep neuralnetwork 220 can be exploited to map a quantitative acquisition tocorresponding contrast image(s) of any target type, using the optimizednetwork parameters.

The method 600 may be repeated a plurality of times to train the deepneural network 520. As such, one set of network parameters is trained onand applied to multiple contrast types. After training and validation,the deep neural network 520 may be used to synthesize any type ofcontrast images identified by the operator.

Referring to FIG. 7 , T1-weighted and T2-FLAIR images of a brainobtained by various techniques are shown. The first column shows groundtruth images acquired by contrast scans as references, T1-weighted imageat the top and T2-FLAIR image at the bottom. The second column showssynthesized images by inter-contrast deep learning approach (e.g., themethod described above with reference to FIGS. 5 and 6 ). The thirdcolumn shows synthesized images by intra-contrast deep learning approach(e.g., the method described above with reference to FIG. 2 ). The fourthcolumn shows synthesized images by conventional model-fitting method.The inter-contrast synthesized images are shown to be the closest to theground truth images because the convolutions share information betweenall channels, allowing them to exploit correlations between the multiplecontrast types of images on a data-driven basis.

Referring to FIG. 8 , a flow chart of a method 800 for synthesizingcontrast image(s) from a quantification acquisition is shown, inaccordance with an exemplary embodiment. At 802, a quantification scanis performed by an MRI system, for example, the MRI system 100 in FIG. 1. As used herein, a “quantification scan” refers to the measurement ofMR signals reflecting absolute values of physical parameters of tissuesbeing scanned. In some embodiments, an MDME sequence may be used toperform the quantification scan. Any other appropriate quantificationsequences may be used to perform the quantification scan.

At an optional operation 804, the target contrast type identified by anoperator is received. This operation is not needed for theintra-contrast method, but may be included in the inter-contrast method.The operator may determine which target contrast type is desired.

At an operation 806, a trained deep neural network is used to synthesizea contrast image from the quantitative acquisition, based on the targetcontrast type if applicable. As used herein, the quantitativeacquisition refers to MR signals acquired by the quantification scan. Insome embodiments, the quantitative acquisition may be the raw MRIsignals in k-space. In some embodiments, the quantitative acquisitionmay be the reconstructed signals in the image domain. For example, whenan MDME with four delay times and two echo times is used to scan, thequantitative acquisition may be eight complex images.

In some embodiments, the deep neural network may be a U-Net CNN with anencoder-decoder structure (e.g., deep neural networks 220 and 520). Insome embodiments, a different neural network architecture may be used,such as ResNets, autoencoder, recurrent neural networks, fully connectednetworks, and so on. The deep neural network has been trained to map thequantitative acquisition to one or more contrast types of images. Forexample, the weights and biases of the convolutional layers of the deepneural network have been optimized based on training datasets. In someembodiments, a plurality pairs of quantitative acquisition andcorresponding ground truth contrast image(s) acquired by contrast scansmay be used to train the deep neural network. In some embodiments wherethe inter-contrast approach is implemented, the deep neural network maybe trained in accordance with the method 600 described above withreference to FIG. 6 .

In some embodiments, the contrast image synthesized by the trainedneural network may then be output, at operation 810. The contrast typemay include, for example, T1- and T2-weighted, T1- and T2-FLAIR, protondensity-weighted, and STIR images. In some embodiments, the contrasttype may be one that cannot be synthesized by the conventionalpixel-by-pixel (or voxel-by-voxel) model fitting method, for example,the T2*-weighted image.

In some embodiments where a discriminator (e.g., as shown in FIG. 5 ) isused, the discriminator may decide whether the synthesized image is ofthe realistic image quality, at operation 808. In some embodiments, thediscriminator is CNN with multiple convolutional layers trained todistinguish images of realistic image quality from those of fake imagequality. If the synthesized image does not have realistic image quality(i.e., “NO” at 808), the indication of fake image quality may be outputat operation 812. In some embodiments, the synthesized image may beoutput along with the indication. If the synthesized image has realisticimage quality (i.e., “YES” at 808), a classifier (e.g., as shown in FIG.5 ) may decide whether the type of the synthesized contrast imagematches the target contrast type, at operation 814. If the types do notmatch (i.e., “NO” at 814), such indication may be output at operation810. In some embodiments, the synthesized image may be output along withthe indication. If the types match (i.e., “YES” at 814), the synthesizedcontrast image is output at 810.

With the systems and methods described above, synthetic MRI withdata-drive deep learning approach can outperform conventionalpixel-by-pixel (or voxel-by-voxel) model-fitting method with improvedaccuracy and reduced artifacts.

In addition to any previously indicated modification, numerous othervariations and alternative arrangements may be devised by those skilledin the art without departing from the spirit and scope of thisdescription, and appended claims are intended to cover suchmodifications and arrangements. Thus, while the information has beendescribed above with particularity and detail in connection with what ispresently deemed to be the most practical and preferred aspects, it willbe apparent to those of ordinary skill in the art that numerousmodifications, including, but not limited to, form, function, manner ofoperation and use may be made without departing from the principles andconcepts set forth herein. Further, it is to be understood thatembodiments of the present invention may be applicable to traditionalMRI, UTE MRI, Silent MRI, PET/MRI, multispectral/hyperspectral imaging,multi-energy CT, multi-tracer PET, and/or any type of MRI based imagingsystem with appropriate adjustment. Further still, as will beappreciated, embodiments of the present invention related imagingsystems may be used to analyze tissue generally and are not limited tohuman tissue. Also, as used herein, the examples and embodiments, in allrespects, are meant to be illustrative only and should not be construedto be limiting in any manner.

What is claimed is:
 1. A method for synthesizing a magnetic resonance(MR) contrast image, the method comprising: performing a quantificationscan, wherein the quantification scan is the measurement of MR signalsreflecting absolute values of physical parameters of tissues beingscanned; using a trained deep neural network to synthesize the MRcontrast image from a quantitative acquisition obtained by thequantification scan; outputting the MR contrast image synthesized by thetrained deep neural network; and wherein the quantification scan is theinput to the trained deep neural network.
 2. The method of claim 1,wherein performing the quantification scan comprises acquiring MRsignals with a multi-delay multi-echo (MDME) sequence.
 3. The method ofclaim 1, wherein the quantitative acquisition is multiple complex imagesreconstructed from MR signals acquired by the quantification scan. 4.The method of claim 1, wherein the MR contrast image synthesized by thetrained deep neural network is one of T1-weighted image, T-2 weightedimage, T1-FLAIR image, T2-FLAIR image, proton density weighted image,and STIR image.
 5. The method of claim 1, wherein the MR contrast imagesynthesized by the trained deep neural network is a T2*-weighted image.6. The method of claim 1, further comprising: receiving, at the traineddeep neural network, a target contrast type for the synthesized MRcontrast image.
 7. The method of claim 6, further comprising: using atrained discriminator to determine whether the synthesized MR contrastimage is of realistic image quality; and in response to determining thatthe synthesized MR contrast image is not of realistic image quality,outputting an indication to indicate that.
 8. The method of claim 6,further comprising: using a trained classifier to determine the contrasttype of the synthesized MR contrast image; determining whether thedetermined contrast type matches the target contrast type; and inresponse to determining that the determined contrast type does not matchthe target contrast type, outputting an indication to indicate that. 9.A magnetic resonance imaging (MRI) system comprising: a gradient coilassembly configured to generate gradient magnetic fields; a radiofrequency (RF) coil assembly configured to generate RF pulses; adisplay; a controller in communication with the gradient coil assembly,the RF coil assembly, and the display and configured to: instruct thegradient coil assembly and RF assembly to generate a sequence to performa quantification scan, wherein the quantification scan is themeasurement of MR signals reflecting absolute values of physicalparameters of tissues being scanned; instruct a trained deep neuralnetwork to synthesize a MR contrast image from a quantitativeacquisition obtained by the quantification scan; output the MR contrastimage synthesized by the trained deep neural network at the display; andwherein the quantification scan is the input to the trained deep neuralnetwork.
 10. The MRI system of claim 9, wherein the sequence for thequantification scan is an MDME sequence.
 11. The MRI system of claim 9,wherein the target contrast type is any of T1-weighted, T2-weighted,T2*-weighted image, T1-FLAIR, T2-FLAIR, proton density weighted, andSTIR.
 12. The MRI system of claim 9, further comprising a memory storingthe trained deep neural network.
 13. The MRI system of claim 12, whereinthe trained deep neural network is a multi-scale U-Net convolutionalneural network (CNN) with an encoder-decoder structure.
 14. A method forsynthesizing a magnetic resonance (MR) contrast image, the methodcomprising: performing a quantification scan, wherein the quantificationscan is the measurement of MR signals reflecting absolute values ofphysical parameters of tissues being scanned; receiving a targetcontrast type; using a trained deep neural network to synthesize the MRcontrast image from a quantitative acquisition obtained by thequantification scan based on the target contrast type; using a traineddiscriminator to determine whether the synthesized MR contrast image isof realistic image quality; in response to determining that thesynthesized MR contrast image is of realistic image quality, using atrained classifier to determine the contrast type of the synthesized MRcontrast image; determining whether the determined contrast type matchesthe target contrast type; in response to determining that the determinedcontrast type matches the target contrast type, outputting thesynthesized image; and wherein the quantification scan is the input tothe trained deep neural network.
 15. The method of claim 14, furthercomprising: in response to determining that the synthesized MR contrastimage is not of realistic image quality, outputting an indication toindicate that.
 16. The method of claim 14, further comprising: inresponse to determining that the determined contrast type does not matchthe target contrast type, outputting an indication to indicate that. 17.The method of claim 14, wherein the target contrast type is any ofT1-weighted, T2-weighted, T2*-weighted image, T1-FLAIR, T2-FLAIR, protondensity weighted, and STIR.
 18. The method of claim 17, wherein thetrained deep neural network uses one set of parameters for all contrasttypes.